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ABSTRACT 

The National Association of State Directors of 
Migrant Education (NASDME) is committed to develop and support an 
educational accountability system that works. A good accountability 
system should: (1) accurately report on the efforts of migrant 
education and the progress of migrant children served; (2) reward or 
sanction the appropriate entities or practices; and (3) result in a 
higher probability that migrant children will succeed in school. 
NASDME offers recommendations to the U.S. Congress and U.S. 
Department of Education as to how the accountability system in 
migrant education can begin to approach these ideals. These 
recommendations are: (1) continued federal support for development of 
a fl exi b 1 e performance indicat or sys tern incorpor at ing Of f ice of 
Educational Research and Improvement's design criteria; (2) 
authorization and funding of high quality evaluation designs, 
especially randomized field experiments, for evaluating program 
effects and program variations; (3) dissemination of the program 
monitoring instrument to the states 1 year in advance of its use; and 
(4) support for an optional peer monitoring and compliance review 
program. In addition, NASDME recommends that its members work to 
improve each other and take responsibility for assessing the quality 
of the migrant education program at an annual leadership retreat. 
Several models for educational accountability are described, 
including performance reporting, monitoring and compliance, incentive 
systems, free market theory, school-based management, and 
professional autonomy and responsibility. (SV) 
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The Argument for Accountability That Works 
EXECUTIVE SUMMARY 



The National Association of State Directors of Migrant Education (NASDME) is committed to 
cooperatively develop and then fully support an accountability system that works. As an Association, 
we believe that an effective system of accountability is important to achieving our mission as migrant 
educators. We also believe that a poorly conceived accountability system, or one whose proper 
implementation is improbable, is not only harmful but also requires our resistance and withholding 
of support. We take the position that the arguments for and about an accountability system for 
migrant education need to be judged on the degree to which the proposed accountability model(s) and 
measures workl 

At its core, educational accountability is about reporting progress publicly, recognizing success 
or failure when it occurs, and encouraging educational institutions to move continuously toward full 
attainment of their stated mission. We believe that a good accountably system should: 

1) accurately report on the efforts of migrant education and the progress of 
migrant children due to migrant education, 

2) reward or sanction the appropriate entities or practices, and 

3) result in a higher probability that the desired state of affairs will be reached, 
that is, helping migrant children to succeed in school. 

As the next reauthorization of the Chapter 1 Migrant Education Program draws near, 

NASDME thinks it is especially important to examine the current accountability system in light of 

these attributes. Now is the time for the United States Congress, the U.S. Department of Education, 

and other education policymakers to consider this issue thoughtfully. We have taken this opportunity 

to state our ideas on how the present accountability system in migrant education can begin to 

approach the ideals presented above. Our thoughts and recommendations are offered in light of the 

Association's continued leadership and collective experience in serving migrant children, their families, 

and in improving the equality of educational opportunity. 
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Recommendation #2: Congress and OME should continue to support the 
development of a flexible performance indicator system for migrant education 
and adopt OERI criteria as the system's design standards. 

NASDME views the development of a flexible performance indicator system to collect and 

report data on desired educational outcomes as one means toward improving the current accountability 

system in migrant education. NASDME recommends that Congress and the U.S. Department of 

Education, Office of Migrant Education (OME) continue to support the development of MENAES 

(Migrant Education Needs Assessment and Evaluation System) as a sub-function of the Migrant 

Student Record Transfer System (MSRTS) to provide all States with the capacity to conduct annual 

evaluations of their own programs. NASDME suggests that the Office of Educational Research and 

Improvement's (OERI) recommended criteria for an effective indicator-based performance reporting 

model be adopted as the design standards for MENAES. NASDME believes that the most effective 

system is one that allows each individual State program to accurately report the level(s) of attainment 

of annual performance objectives. 

Recommendation #2: Congress and OME should authorize and encourage the 
use of high quality evaluation designs, especially randomized field 
experiments, for evaluating program effects and program variations. 

The .ne of higher quality evaluation designs is often mentioned, but is seldom supported 
actively. The obstacles to conducting good "impact" evaluations are many, and are often valid. 
NASDME agrees that these studies are difficult to execute and cannot be routinely implemented. But, 
given the intense interest in discerning program effects, NASDME urges that this type of evaluation 
be encouraged and supported in all States where the evaluation design is feasible. NASDME suggests 
that the use of higher quality evaluation designs is in keeping with appropriate professional standards 
and with public interest in evidence of this kind. 

Authorizing high quality program evaluations, however, is only one facet of this 
recommendation. To support better evaluation designs that guide program improvement and estimate 
program effects, NASDME asserts that in the absence of political and fiscal support, the effort and 
risk-taking that is necessary to employ thesa designs will not occur. New evaluation efforts can not 
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compete for a larger portion of an already limited set of resources. Especially, when the competition 

is "serving children." NASDME suggests that additional funds, beyond the regular allocation of funds 

provided by the statute (i.e., Sections 1201 and 1203), be made available to all States during the five 

year period coinciding with re-authorization of the law. The new funds would be used to support 

program evaluations that meet the standards of a high quality design. 

Recommendation #3: OME should disseminate and discuss the program 
monitoring instrument one year in advance of its use in on-site program 

reviews. (seenote> 

NASDME recommends that OME disseminate the official program monitoring instrument to 

the States at least one year in advance of its use. The monitoring guide provides the States with a 

useful tool to help direct (and re-examine) on-going efforts in program development and program 

compliance. By allowing the States to preview the guide, to ask questions, and to better prepare for 

monitoring visits, OME will prompt precisely the type of action that it hopes to encourage within every 

State program. The dissemination of the guide should be accompanied by written and oral 

presentations on 1) the findings from the previous year's monitoring activities and 2) on those aspects 

of the program that OME will consider a priority area(s) for review in the upcoming year. NASMDE 

suggests that the full dissemination and discussion of the audit process will do more to stimulate 

greater accountability than trying to catch States unaware. The Association believes that the success 

of a monitoring and compliance approach should not be measured by the number of audit exceptions, 

but rather by the absence of errors and by the improvements States make in order to minimize the 

number of mistakes or lapses. 

Recommendation #4: Congress and OME should support an optional peer 
monitoring and compliance review program. 

To improve the functionality of program monitoring and compliance reviews, Congress and 
OME should consider supporting an optional "peer review" for the migrant education program. The 
peer monitoring and compliance review option would enable a State Director to identify three peers 
to conduct a week-long review of the State's program using the OME monitoring instrument as a 
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guide. At the end of the review, the peer group would conduct a formal exit interview with the State 

Director and would later follow up their presentation of findings in a confidential written report. The 

reviews could be scheduled for up to one and one-half years before the next scheduled OME review 

or one and one-half years after the most recent monitoring visit. 

The benefits of this approach arc three-fold. First, the frequency of program monitoring can 

be increased and thus strengthen the cycle ofcorrective feedback (and the perception of its value), The 

program monitoring cycle would now provide for a low-risk "learning" experience that would better 

help to identify operational difficulties and to evaluate potential solutions with experienced colleagues. 

Thus, the second major benefit of this approach is the immediate access to respected peers who 

understand the problems first-hand and who are in a position to criticize constructively without 

inducing penalties. The third benefit of the peer review process is that it would be an invaluable 

learning experience for the members of each review team. Over time, the cumulati/e effect of the 

process will sharpen NASDME's collective wisdom. The ongoing official program reviews would 

continue to provide OME an opportunity to publicly monitor and accredit each State agency's efforts 

in the provision of services, to review management control systems that influence program quality, 

and to examine compliance with program standards or regulations. 

Recommendation #5: NASDME should actively help its members improve each 
other and take responsibility for assessing the quality of the migrant 
education program at an annual leadership retreat. 

The desire to innovate, to develop, and to implement high quality services for migrant children 
is rooted in the inspiration and commitment that migrant educators bring to the profession and that 
we demand of each other. The desire to persist until the equality of educational opportunity is secured 
for all migrant children is sustained largely by a set of beliefs about the promise of migrant children 
and the importance of helping them to realize their full potential. The desire to embrace these 
challenges and to take risks to serve migrant children better is part of a culture that must exist within 
the community of migrant educators-if the program is to succeed. Accountability based on technical 
solutions (i.e., reporting numbers and compliance) can only function if accompanied by accountability 

iv 
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driven by professionalism. A good accountability system for migrant education must include, 
therefore, the means for progressive change in the professional roles, norms, and values of migrant 
educators. NASDME believes that it is through this sort of learning that it could stimulate a self- 
renewing process of reflection, problem-solving, and action that leads to accountability of the most 
fundamental kind. 

*** 

Throughout the past twenty-five years, the federal role in education has been to advocate for 
the interests of those children who are most in need: the poor, the handicapped, racial and ethnic 
minorities, and those children who by their life's circumstance were often excluded from the full 
benefits of the public education system. From NASDME's perspective, the 1990s have witnessed the 
emergence of a new federal role in education. In addition to its traditional functions, the federal role 
in education now incorporates a growing and active interest in promoting education reform. 

NASDME views the Chapter 1 Migrant Education Program as an important contribution to 
both of the federal government's goals in education: equality of educational opportunity and education 
reform. We believe that a migrant education program operating with an effective accountability 
system can help to increase progress toward these two goals. In this position paper, we present five 
important improvements to the existing accountability system. The changes do not call for wholesale 
transformations, but rather propose small well-focused shifts in emphasis in current accountability 
policies and practices. We believe that the proposed changes will help to create a high leverage 
accountability system, and thus conceived and implemented, one that will ivorkl 
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The Argument for Accountability That Works 



Aepared by Alexander Goniprow 
on the behalf of the 
National Association of State Directors of Migrant Education 

No one questions the public's right to hold educators accountable for the work we accomplish 
with resources from the common wealth. People have a right to know what kind of results are being 
produced with public funds. Educator, overseer, and client all hope that accountability measures will 
weed out ineffective schools and programs and leave the productive ones, healthy and unharmed. 
Most people believe that accountability systems should also help to stimulate on-going progress-so 
that what "ought" to be done is eventually accomplished. At its core, educational accountability is 
about reporting progress publicly, recognizing success or failure when it occurs, and encouraging 
educational institutions to move continuously toward full attainment of their stated mission. 

One problem with accountability in the 1990s is that it too often fails to achieve any or all of 
these purposes. Most of the current accountability schemes in education do not work-not because of 
institutional resistance-but because of serious weaknesses in the accountability models and 
performance measures employed. And, when an accountability system does not work, it often serves 
unfortunately to obscure reality, to waste resources, and to bludgeon all educators rather than to serve 
as a constructive tool of social policy setting and education planning strategy. 

The National Association of State Directors of Migrant Education (NASDME) 1 is committed 
to cooperatively develop and then fully support an accountability system that works. As an 
Association, we believe that an effective system of accountability is important to achieving our mission 
as migrant educators, We also believe that a poorly conceived accountability system, or one whose 
proper implementation is improbable, is not only harmful but also requires our resistance and 
withholding of support. In this paper, we take the position that the arguments for and about an 
accountability system for migrant education need to be judged on the degree to which the proposed 
accountability model(s) and measures ivorkl 
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The Argument for Accountability that Works 

We believe that a good accountability system should: 

1) accurately report on the efforts of migrant education and the progress of migrant children due 
to migrant education, 

2) reward or sanction the appropriate entities or practices, and 

3) result in a higher probability that the desired state of affairs will be reached, that is, helping 
migrant children to succeed in school. 

As the next reauthorization of the Chapter 1 Migrant Education Program draws near, it is 
especially important to examine the current accountability system in light of these attributes. Now 
is the time for the United States Congress, the U.S. Department of Education, and the public to 
consider this issue thoughtfully. We use this paper to present our ideas on how the accountability 
system can begin to approach these three ideals and further, how it can be developed. Our thoughts 
and recommendations are offered in light of the Association's continued leadership and collective 
experience in serving migrant children, their families, and in improving the equality of educational 
opportunity. 

We begin by briefly summarizing what others have written about accountability: its definition, 
the theory that guides its popular appeal, and the current "state-of-the-art." We do so to establish the 
context for our subsequent discussion and recommendations. After framing the topic, we explore why 
the present strategy for accountability in migrant education needs improvement. We then offer our 
specific recommendations for building a better and more effective accountability system. Lastly, we 
conclude by suggesting how the Congress and the education policymakers might better view the role 
of migrant education. A proposal if you will, on how migrant education could help improve knowledge 
and understanding of accountability systems in education more generally. 
THE CONCEPT OF ACCOUNTABILITY 

The current definition of accountability is thought to have emerged during the 1660s when the 
definition "to render an account of, to explain and to answer for" came into use (Wagner, 1989, p. 2). 

Page 2 
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Tlie Argument for Accountability that Works 

As the definition of accountability evolved it began to incorporate the notion of role responsibilities in 

organized relationships where "individuals with particular duties were liable to review and sanctions 

if their actions failed to satisfy those in authority" (Kogan, 1986, p. 18). Thus the common definition 

of accountability meant "the requirement on the occupanu of a role, by those who authorize that role, 

to answer for the results of the work expected from him in the role" (Newman and Rowbottom, 1968, 

p. 26). Browder (1975) noted another aspect of the common form when he highlighted the benefit of 

"operationally-specifying" what tasks are to be accomplished. 

The heart of the process is for the party "standing to account," the steward, to explain 
as rationally as possible the results of efforts to achieve the specified tasks of objectives 
of his stewardship (p. 8). 

Because the description of what performance is expected is so specifically written, the 
steward's obligations are more pointed. He knows what is expected and what is not 
expected.... Similarly, the agreement also obligates the reviewer by pre-establishing 
the criteria of his expectations. He cannot whimsically change his expectations in 
midstream, add "surprise" responsibilities, or otherwise escape his own responsibility 
to define what he expects to happen before the steward begins the task (p. 9). 

Up to this point, the responsibility for the specification of tasks was primarily ceded to those in 

authority. During the early 1970s, however, Lessinger (1970), Browder (1971), Alkin (1972), and 

others began to see the specification of tasks more as the product of a "negotiation" process, especially 

in the public sector. Lessinger (1970) advocated a negotiation process that resulted in forma! written 

contracts with explicitly stipulated the outcomes, conditions, and performance standards. 

At its most basic level, it [accountability] means an agent, public ot private, entering 
into a contractual agreement to perform a service will be answerable for performing 
according to agreed-upon terms, within an established time period, and with a 
stipulated use of resources and performance standards (from Lessinger as cited in 
Browder, 1975, p. 12). 

Others, like, Lopez (1970), placed a somewhat less extreme emphasis on formal contracting. 

Accountability refers to the process of expecting each member of an organization to 
answer to someone for doing specific things according to specific plans and against 
certain timetables to accomplish tangible performance results (from Lopez as cited in 
Browder, 1975, p. 12). 
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The Argument for Accountability that Works 



One of the first applications of accountability in education (in the 20th century) occurred when 

Senator Robert Kennedy insisted that the Title I Elementary and Secondary Education Act of 1965 

include an evaluation component. The evaluation component required local projects to adopt 

procedures to measure objectively the effectiveness of their programs in meeting the needs of 

educationally deprived children. In reviewing the impact of this evaluation requirement, Millsap 

(1984) cited McLaughlin's work on evaluation and refom and wrote: 

The law contained no language that this information would be used for any particular 
purpose, and certainly none that pointed to evaluation as a means of school reform. 
Yet, Kennedy viewed mandated evaluation as a means of political accountability. He 
hoped that project reports would provide parents with the information (and thus the 
power) they needed to ensure that Title I dollars were spent in the effective interests 
of poor children (p. 10). 

Although Kennedy was eventually disheartened with the results of his evaluation plan, the 
hypothetical relationship between the release of information, the generation of corrective feedback, and 
school improvement gained increasing currency throughout the ensuing years. By the mid 1970s, the 
public's concern over rising costs in education and the perceived lack of results coalesced with an 
idealized perception of the private sector's success in getting "bottom line" performance. The result 
of this convergence was a call for more business-like accountability in education. 

Leon Lessinger, an associate commissioner with the U.S. Office of Education, became a leading 
proponent of the business approach to educational accountability. He envisioned the techniques of 
modern business and management theory as appropriate and effective means of improving educational 
results. 

In essence, these procedures that have worked well in certain areas emphasize: 
clarifying goals, designing plans for attaining them, measuring progress toward them, 
diagnosing difficulties, and redesigning the plan, with subsequent appraisal, and 
further revisions, if necessary. In applying strategic planning, the total organization 
and its several parts are held accountable for reaching the goals. As these 
management procedures are becoming more widely known, they are frequently being 
recommended by businessmen and other laymen for use in schools (Lessinger and 
Tyler, 1971). 
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The Argument for Accountability that Works 

The addition of the business world's problem-solving sequence solidified the present-day definition of 
educational accountability. Like its kin in the private sector, educational accountability now drew 
upon "control theory ," where control was defined as "the process of specifying preferred states of affairs 
and revising ongoing processes so as to move in the direction of these preferred states" (Etzioni, 1968, 
p. 45). Educational accountability became both a product and a process. To simply account, or to be 
held answerable, was insufficient. A good educational accountability system now included a process 
that helped to lead to the full attainment of stated aims. 
MODELS FOR EDUCATIONAL ACCOUNTABILITY 

In July of 1990, the U.S. Department of Education published Michael Kirst's Accountability; 
Implications for State and Local Policymakers. The paper offers an excellent summary of six 
contemporary approaches to educational accountability. The accountability strategies broadly 
represent the "state-of-the-art." The authors of the report believe that a sensible combination of these 
models, tailored to a specific State or local context, could create an effective accountability system. 
Simultaneously, the authors neither oversell nor oversimplify the very real and current limitations 
in most of the models. For the reader, each of the approaches i* briefly described below, both in terms 
of its application as well as its cunent development. 
Accountability as Performance Reporting 

The performance reporting model has received the greatest amount of attention and 
developmental effort in the past decade. Performance reporting requires the specification and 
measurement of desired educational outcomes and the reporting of whether or not the outcomes have 
been attained. The information usually comes in the form of psychometric indices or performance 
objectives that are evaluated by particular interest groups (i.e., parents, staff, school boards, etc.) and 
the tax-paying public at-large. In essence, the performance reporting accountability model is "a set 
of indicators or statistics that provide information about how well schools are performing" (OERI 
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The Argument for Accountability that Works 

Study Group, 1988, p. vii). It assumes that the information is reliable, fair, and useful for improving 
schools. The model also assumes, that if the results are poor, the information itself will stimulate and 
arouse parent groups or other stakeholders to demand improvements or changes in the education 
services. 

In 1988, an OERI State Accountability Study Group specifically focused on this model in their 

report, Creating Responsible and Responsive Accountability Systems. The authors write that the 

indicators in a performance reporting accountability model should: 

o Measure the central features of schooling (i.e., inputs, outputs, and 
implementation); 

o Measure what is actually being taught or considered important for students 
to know; 

o Provide information that is policy relevant; 

o Focus on the school site; 

o Allow for fair comparison; and 

o Maximize the usefulness of the data collected (i.e., provide information that 
can be readily understood by the audience) and minimize the burden of 
collecting it (OERI Study Group, 1988, p. vii). 

The report also cautioned readers interested in performance reporting that there were several 

unresolved dilemmas in developing and implementing the model. Specifically, the authors noted six 

concerns: 1) balancing the model's oversight and improvement functions; 2) determining the 

appropriate level of accountability (school, district, or State); 3) balancing desire for statewide 

comparability with local district needs in determining the information to be collected; 4) expanding 

alternatives to traditional standardized tests; 5) making unfair comparisons between schools or 

programs that operate in vastly different contexts; and 6) ensuring that the agencies themselves have 

adequate capacity to collect and use data. 
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The Argument for Accountability that Works 

While each of these issues does deserve further attention, Terry Petersen, the chair of the 

Study Group, rightly focused his remarks on the central concern with this model-the indicators. 

Educator displeasure is often not about the principle that schools should be held 
accountable. Rather it stems from their legitimate concern about the accountability 
measures selected. These measures may focus education too narrowly, may cause 
schools to pay attention to rote learning only, may encourage teaching to test items, 
or may be inappropriately used (OEM Study Group, 1988, p. iii). 

In fact, the group strongly recommended the use of multiple indicators to improve the design and use 

of the performance reporting model. 

Whether an accountability system is used to reward high performance, remedy poor 
performance, or inform an overall school improvement process, it should collect data 
on multiple indicators to provide an integrated picture of the schooling environment 
(OERI, 1988, p. 46). 

The report suggested a number of possible alternative indicators: the amount of time spent on 
different subjects; student attendance; promotion and dropout rates; percentage of students meeting 
entrance or graduation requirements; changes in student attitudes and social behavior; and broadly 
defined student achievement. 

Academic achievement received additional attention; specifically, the group commended those 
States that were experimenting with enhancements of the traditional standardized achievement tests 
and looked forward to other supplements or alternatives to standardized testing that are now in the 
research and development stage. The need for better alternatives to standardized and norm-referenced 
achievement tests is supported and echoed in the avalanche of recent reports that question the utility 
of the existing instruments (see, for example, Testing in Chapter 1: Issues and Options, Turnbull, 
1991). The OERI report also suggested that data on teacher quality, fiscal resources, administrative 
leadership, curricula, local assessment, evaluation and planning systems, and community support be 
collected to understand the schooling context more comprehensively and accurately. 

In sum, the importance of the performance reporting model in any accountability system is 
plainly evident. The theoretical design and application of the model are consistent with emergent 
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The Argument for Accountability tfiat Works 

definition of accountability. In addition, all of the other models will need to rely on it to some extent 
for them to work. Nevertheless, poorly designed performance reporting models must be avoided as 
they may "produce unintended effects that may constrain instruction and shape administrative policies 
in ways that many find inappropriate" (OERI, 1988, p. vi). In the future, the potential of the 
performance reporting model can be realized, but only if its present limitations are understood and 
attended to by its users. 

Accountability through Monitoring and Compliance with Standards or Regulations 

The use of monitoring and compliance reviews to ensure accountability stems from public fears 
that people are not told the full truth about education program quality, or that resources are being 
used improperly. The approach employs either internal or external program auditors to examine and 
report on school or program operation. The intent of such audits is to monitor the agency's efforts in 
the provision of services, to review management control systems that influence program quality, and 
to examine compliance with program standards or regulations. 

In the U.S. Comptroller General's Standards for Audit of Government Organizations, Programs, 
Activities, and Functions (1981), the three elements of auditing are defined as: 

1. Financial and compliance-determines (a) whether the financial statements of 
an audited entity present fairly the financial position and the results of 
financial operations in accordance with generally accepted accounting 
principles and (b) whether the entity has complied with laws and regulations 
that may have a material effect upon the financial statements. 

2. Economy and efficiency-determines (a) whether the entity is managing and 
utilizing its resources (such as personnel, property, space) economically and 
efficiently, (b) the causes of inefficiencies or uneconomical practices, and (c) 
whether the entity has complied with laws and regulations concerning matters 
of economy and efficiency. 

3. Program lesults-determines whether the desired results or benefits 
established by the legislature or other authorizing body are being achieved and 
(b) whether the agency has considered alternatives that might yield desired 
results at a lower cost (p. 3). 
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The Argument for Accountability that Works 

The U.S. Comptroller General advises authorities to only select one or two aspects of an audit and 
then, only if it will meet the needs of expected users of the audit results. It is not intended, nor is it 
feasible or desirable that every audit include all three elements. 

In most cases, audits are conducted at the program site. The auditors' activities are governed 
by an audit plan that details the scope of their work. Once the review is completed, the program 
auditor usually conducts an "exit" conference to brief the top administrator on the major findings, 
commendations, and recommendations. A formal report is written later by the program auditor and 
sent to the appropriate parties for response and follow-up. Dissemination of the report to the public 
and/or other authorities is expected to ensure that all corrective actions or improvements will be 
undertaken. 

The effectiveness of this model depends on the frequency of reviews, the expertise of the 
program auditors, agreement on the activities or functions to be monitored and the performance 
standards to be met, and the type of rewards and sanctions that follow at the report's release. In 
education, accreditation, financial, and compliance audits, all of which carry the possibility of severe 
sanctions, are more likely to hold the schools accountable and initiate change than is generic program 
momcoring. 

Accountability through Incentive Systems 

Instead of using both the carrot and the stick, the incentive system model focuses solely on the 
provision or withholding of rewards only. The underlying concept of an incentive model is "to reward 
for results that are guided by incentives designed to provide inducement for specific actions by 
educators" (Kirst, 1990, p. 8). Examples of the incentive model are performance contracting, merit 
pay, bonuses, promised deregulation of the school site, and other nonmonetary award strategies. 
Critics of this approach are quick to mention an experiment in the 1960s where a contractor in 
Texarkana, Texas falsified test data in order to make more money (Kirst, 1990). Additional resistance 
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The Argument for Accountability that Works 

to this approach has come from teacher unions that question the ability of such systems to 
discriminate fairly between the performances of teachers or schools, especially when the student 
populations come from varying backgrounds. Not surprisingly, a significant obstacle to implementing 
this accountability model is achieving consensus on fair and valid indicators of teacher or school 
performance. 

Accountability through Reliance on the Market 

Consumer choice is the key concept in the accountability model that relies on free market 
theory. Through various means such as vouchers, tuition tax credits, and open enrollment, this 
approach enables parents to choose their children's school. Accountability occurs "when consumers 
choose between schools, with the *bad schools' presumably closing if pupils leave" (Kirst, 1990, p. 9). 
To be effective, the school choice model assumes that parents will have sufficient information to make 
these decisions, that "good" alternative schools are available, and that the marginally performing 
public schools will have the wherewithal to improve before going out of business. 
Accountability through Changing the Locus of Control 

A school-based governance and management structure is at the center of this accountability 
model. It requires central office administrators to decentralize the control of education policy to each 
local school site. In a school-based management system, the school is the primary unit of educational 
decision-making (Lindelow, 1984). The system may rely on either the site administrator, a cadre of 
teachers, a parent group, or a school-site council (that usually includes administrators, teachers, and 
parents) to exercise decision-making powers. Notwithstanding the precise format of the governing 
structure, in this system, the leadership group reports and is answerable to members of the local 
community. 

Changing the locus of control from a distant central authority to the school site *s intended to 
improve accountability in two ways. First, if people at the school site decide how the school should 
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operate, without interference from the centra] office, then the school can be held directly accountable 
for the results. Second, the model presumes that an individual school site will be more responsive to 
parents and the local neighborhood than district level bodies. The hope, of course, is that parents and 
community members will be able to exert greater influence on the decision-making process when 
expectations are not being met and thus prompt improved results. 
Accountability through Changing Professional Roles 

The sixth approach to accountability considers "professionalism" and "stresses the desirability 
of accountability coming from within the teaching force rather than being imposed by external 
authorities 11 (Kirst, 1990, p. 24). The "key is for teachers to help each ocher improve and to take 
responsibility for assessing quality" (Kirst, 1990, p. 25); this is accomplished by giving teachers greater 
discretion and autonomy in the classroom. The model presumes that teachers, treated as 
professionals, will have more ownership and commitment to educational outcomes developed and 
evaluated with their colleagues. The model also recognizes the value in developing and enhancing 
school administrators' management sophistication and leadership skills. Ultimately, in this model, 
educational accountability is sustained by self-evaluation and increased professionalism. 

Each of the six accountability models discussed above has its own strengths and weaknesses. 
A good accountability system will draw upon each of the models, and integrate them in creative and 
effective ways. To a great degree, each relies on performance indicators to inform judgements that 
need to be made in determining who to reward, who to sanction, and whether or not expectations have 
been met. Some of these strategies are more appropriate than others for migrant education programs. 
For example, given the mobility of the migrant population, the "consumer choice" model and the "locus 
of control" model are much less feasible. 
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In the next section, we will proceed to describe and evaluate the current combination of 
accountability models used in the Chapter 1 Migrant Education Program-and we will recommend 
improvements concurrently. 
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CURRENT ACCOUNTABILITY EFFORTS IN MIGRANT EDUCATION 

The accountability system in migrant education begins when an individual Stat? enters into 
an agreement (i.e., the Annual Program Plan) to perform certain services that are required and/or 
permitted by federal law or regulation. Whereas the required services are usually not negotiable, the 
manner in which they are implemented and other permissible activities are negotiated between the 
individual State and the U.S. Department of Education, Office of Migrant Education (OME). An 
acceptable Annual Program Plan should identify the specific objectives and practices that the State 
will be answerable for performing within an established time period, with a stipulated use of 
resources, and according to an agreed-upon set of performance standards. Figure 1 presents an 
example of several performance objectives that typically form the basis of these negotiated agreements. 
FIGURE #1: Sample of Desired Outcomes 



Mission: 


To help migratory children succeed in school. 


Goal: 


To provide supplemental instruction that assists school-age migratory 
children to attain grade level proficiency in reading and writing. 


Performance 
Objective: 


S'jhool-age migrant children with identified deficiencies in reading and 
writing will gain at least one NCE in basic and advanced skills in the 
aieas of deficiency as a result of their participation in at least 90 days 
of the Chapter 1 Migrant In-school Tutorial Project. 

or 

School-age children with identified deficiencies in performing a reading 
and writing task at the appropriate grade level will demonstrate at 
least one level of growth on a four point analytic scoring scale as a 
result of their participation in at least 90% of the Chapter 1 Migrant 
Summer Project. 


Activities: 


The project will employ certified teachers at a ratio not to exceed 1 
teacher for every 6 students. The instruction shall... 



In the current accountability system, there are not many extrinsic rewards for individual 
States that accomplish their specified ends. Of the few tangible rewards, public recognition as an 
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"exemplary" program is the most common. The withholding of program funds, the return of public 
monies, the assignment of personal culpability to an individual(s), and a lowered trust in the State's 
administration of the program are the main sanctions or costs to States that fail to meet their 
obligations. The system is designed to foster pro^ am improvement primarily via reaction to the 
information produced from selected accountability mechanisms and State avoidance of the above- 
mentioned sanctions. 

As depicted below, the current system in migrant education draws upon three of the six 
accountability models for those mechanisms. Although the system heavily emphasizes performance 
reporting, it does attend mode; ..ely to monitoring and compliance reviews as well as to the 
strengthening of professional roles. 

FIGURE 2: Models Used In the Accountability System For Migrant Education 



Performance 

Reporting 

Model 



Monitoring & 

Compliance 

Model 



Professional 
Roles Model 



In the sections that follow, each model of the current accountability system is discussed and 
evaluated individually. Immediately following the discussion of each model, NASDME will recommend 
changes to improve the model's efficacy, and by extension improve the overall system of accountability 
for migrant education. 

Accountability through Performance Reporting in Migrant Education 

The accountability system in migrant education uses two forms of performance reporting. The 
first form involves program evaluation. At a minimum, each State is required to make public the 
results of a biennial evaluation report that assesses the overall progress of participating migratory 
children in grades 2 through 12 (see CFR 34, Subpart E, sections 201.51 - 201.56). The evaluation is 
expected to comply with technical standards that include: representativeness of evaluation findings, 
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reliability and validity of evaluation instruments, soundness of evaluation procedures, and valid 
assessment of project outcomes. Basically, these regulations envision a summative evaluation that 
facilitates reasonable judgements about program outcomes and the academic achievement of migrant 
students. Notwithstanding this primary intent, the States are also expected to use the results of these 
evaluations to determine program effectiveness, to strengthen program planning, to identify areas for 
program improvement, and to improve service quality. 

Past State program evaluations were usually of very modest design. Following the lead of the 
Chapter 1 Basic program, most State agencies attempted to aggregate the calculated difference in pre- 
test and post-test scores of participating migrant students to estimate academic gains over a one-year 
period or from project inception to termination. Commercially published standardized norm-referenced 
achievement tests ai:d state/program supported criterion-referenced skills tests have been the most 
frequently used types of instruments to measure learning. The most popular evaluation design of this 
type is the TIERS Model A design (Title I Evaluation and Reporting System) which relies on 
normative data to substitute for data of a suitable comparison group. 

Tbe second form of performance reporting in migrant education entails the submission of the 
Annual Performance Report (see CFR 34, Subpart E, section 201.51c). This report includes summary 
data from the evaluation efforts mentioned above, and demographic data on the age, gender, race, 
handicapped status, and grade-level of the children served by the program. The primary intent of this 
report is to provide the Secretary of the U.S. Department of Education with statistics :o profile the 
national scope and impact of the program. 

At first glance, the requirements for both program evaluation and the annual performance 
reporting seem reasonable and compatible. The purposes of the reporting requirements are clear and 
address important functions. However, it is the position of NASDME that the current requirements 
for program evaluation and the annual performance report may create a number of unintended and 
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undesirable effects. In short, we believe that their present emphasis on norm-referenced test scores 
will lead to two problems in performance reporting: one, the regulations emphasis undermines support 
for high quality program evaluation; and two, their current emphasis discounts the use of other data- 
data that are necessary for meaningful program improvement at the State level. An unfortunate 
consequence of this influence would be the diminished likelihood of installing a truly effective 
performance reporting model for migrant education. We are also very concerned that an 
unrepresentative supply of norm-referenced test scores, aggregated nationally, without heed to service 
context, may be reported in an inappropriate form, and may subsequently be used by Congress and 
others to make ill-advised policy and political judgements about the migrant education program. 

One element of our concern hinges on the very subtle message (about program evaluation) that 
is advanced by the emphasis on norm-referenced test data. The "NCE" gain score (normal curve 
equivalence), and the interest with which scores alone are pursued by the Congress and U.S. 
Department of Education, suggest that the effects of a program, or the information needed to make 
decisions about a program, can be reduced to one simple statistic. NASDME is particularly worried 
about that message. The current crop of evaluation reports typically contain a simple listing of 
academic gains by grade level (for the students with two sets of scores) and a brief presentation of 
descriptive information on the type of services, location of projects, and number of program 
participants. Few evaluation reports present extended analyses or results from other types of indices. 
Why should States pursue more elaborate and time-consuming evaluation activities, when one 
indicator will suffice? Given the weight accorded that one indicator, is it surprising that very few 
program evaluations try to examine the relationships between a complex treatment and its outcomes? 

The problem of obtaining norm-referenced test data represents another part of our concern. 
In recent years, migrant educators have noted wide-spread problems in the collection and supply of 
test scores. In a recent examination of the student test files in the Migrant Student Record Transfer 
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System (MSRTS) conducted in January 1992, the records indicated that less than 20% of migrant 
students (grades 2-12) have any norm-referenced achievement test scores. 2 Less than 5% of the 
population have matched pretest and posttest scores (C. Nowell, personal communication, March, 5, 
1992). 

The low number of available test scores reflects not only the lack of data input on the part of 
many State programs, but many other factors as well. Efforts have been made to facilitate and 
increase the reporting of test scores. NASDME implemented the Migrant Education Needs 
Assessment and Evaluation System (MENAES) in 1991 to enable the States to input norm-referenced 
test scores, to allow thb national database to match student pre-tests and post-tests (regardless of test 
administration location), and to provide each State with achievement test scores from two data points. 
To some degree this new system has helped, but it is not likely that the majority of States will ever 
succeed in obtaining a sufficient number of tests scores to support a performance reporting system that 
requires large amounts of test data to maintain its public credibility. 

The chief culprits in the low supply of test scores are student mobility, short-duration project 
designs, state testing policies, new assessment practices, and inadequate funding. For over two 
decades, the high mobility of migrant students and their families has frustrated migrant educators' 
efforts to collect test data at two points-in-time. The data collection problem begins after migrant 
children, who are in school and enrolled in a migrant project, are pre-tested. Given the livelihood of 
migrant families, many of the students move with their parents (as they seek work) and do not return 
to the same State— much less the same school building— after their parents work cycle is completed. 
In this common scenario, the initial sending school is not able to obtain a post-test score to complete 
the evaluation activity. Residency patterns are also very transitory for settled-out migrants and they 
lead to the same problem-no post-test scores. 
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A second reason for the low supply of test scores is that migrant education projects are 
designed to accommodate life in the migrant streams. For example, each Spring a number of children 
migrate north as their parents seek seasonal agricultural work. Many of the northern States operate 
summer projects to provide instructional and supportive services to the migrant students in residence 
during the summer months. The services of a summer project are intensive but short. The 
administration of a norm-referenced test for evaluation purposes in this type of project is wholly 
inappropriate. Yet, because the migrant families were moving in the late Spring and/or returning in 
the Fall, the sending districts were also unable to administer a test. In this case, the interaction of 
the project design and family mobility leads to an unintended consequence-no test data. 

Another factor limiting the supply of test scores is that, throughout the country, there are gaps 
in the testing programs of many States and school districts. Some States and local school districts do 
their testing only at specific grade intervals (such as grades 4, 8, and 12) rather than in all grades. 
In addition, many migrant students do not take these tests due to their limited English proficiency. 
Thus, the ideal and certainly the most efficient option-obtaining student test data from the regular 
school program-does not appear to be a real possibility. 

A fourth and growing barrit>. in the future supply of norm-referenced test scores is the 

increasing dissatisfaction amongst educators, in general, with these tests as assessment and 

evaluation instruments. The major objections to these tests are "contentions that they give an 

incomplete or misleading picture of student accomplishments, create incentives for poor educational 

practice, and are especially inappropriate for use in the primary grades" (Turnbull, 1991, p. 9). 

Traditional norm-referenced, standardized tests, which are designed primarily to 
differentiate or rank order students efficiently, are only able to provide information 
about the performance of students in relation to the norm group. While standardized 
tests and norm-referenced scores have many limitations (Cannell 1987; 1989), three 
are particularly serious in relation to the setting of reading literacy goals and the 
assessment of student progress; (1) normative scores are unable to describe what 
students are able to do; (2) normative scores cannot be used to adequately assess 
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student growth,; and (3) normative scores do not maintain constant meaning over time 
(Ivens and Koslin, 1991, p. 4). 

As a by-product of the school reform movement in the 1980s, many schools and programs have already 

re-designed curriculum, have begun to use alternative forms of assessment, and are exploring the use 

of other educational outcomes in conducting program evaluation. Even our national policymakers 

recognize the inadequacias in traditional standardized tests as they call for the development of better 

assessment p aasures to help attain the new and higher education standards envisioned in America 

2000: An Education Strategy (U.S. Department of Education, 1991 revised). 

The decreasing supply of tr ,t scores is also exacerbated by a fifth factor that distinctively 

operates in migrant education, and perhaps in any program affected by diminishing resources. During 

the 1980s, the per-pupil allocation of federal funds to the migrant education program has decreased 

in real dollars from $629 to $260 in 1990 (IMEC, 1991, p. 16). In this environment, migrant educators 

are faced with a dilemma: to spend limited resources on new assessment practices that we believe will 

help migrant children to learn and progress, or to spend resources on instruments that will not meet 

our needs and may not even address the reporting needs of federal education officials or the U.S. 

Congress? We believe that using our resources to address the first challenge represents a better, more 

informed choice. 

Let us step back for a moment. Given the concerns we have outlined about norm-referenced 
tests, what does that mean for a performance-based reporting system? Is the use of non i-referenced 
test scores a prerequisite to a good performance reporting model? Will the expenditure of resources 
and effort to collect norm-referenced test scores increase the value and utility of the performance 
reporting model to the States and to federal officials as well? Will the lack of test scores prohibit 
Congress from making judgements about the merit of the migrant education program? 

We clearly do not think so. NASDME has presented its concerns about the costs of a defacto 
universal testing policy, from both an evaluation design and data supply perspective. There remains 

Page 19 



26 



The Argument for Accountability that Works 

yet a third concern. By universally determining a performance measure prior to the assessment of 

student needo and subsequent program development, the current accountability system ignores a 

critical component. A performance measure should be selected because it will fairly assess what is 

being taught and learned and not the opposite— i.e., at present, because the measure has already been 

selected, the net effect is to design an intervention that will produce a result on that instrument. In 

many cases, norm-referenced tests are appropriate measures because they match pre-established 

curricular objectives. The objectives of migrant education services are, however, often ill-fitted to 

norm-referenced tests. Nonetheless, if a norm-referenced test score is the only performance indicator 

given any credence, it is easy to imagine that districts and States will curtail efforts to match the 

curriculum to student needs in order to find curricular that matches the tests. This result would be 

particularly unfortunate in an era where we are striving to imagine and design more effective 

supplemental educational interventions. NASDME therefore views the development of a more flexible 

system as ideal. A good performance reporting model would incorporate data collection and reporting 

of varied indicators of desired educational outcomes with support for high quality evaluation designs. 

Recommendation #1: Congress and OME should continue to support the 
development of a flexible performance indicator system for migrant education and 
adopt OERI criteria as the system's design standards . 

NASDME is ready to continue its collaborative work in developing a flexible 

performance indicator system. NASDME recommends that Congress and OME 

continue to support the development of MENAES (Migrant Education Needs 

Assessment and Evaluation System) as a sub-function of the MSRTS to provide all 

States with the capacity to evaluate their own programs annually. Achieving a 

national consensus on specific outcome measures is not essential. What is important 

is implementing a system that allows each individual State program to accurately 

report the level of attainment of annual performance objectives. 
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For example, MENAES will continue to support States that wish to report 
norm-referenced test scores to measure what is being taught or considered important 
for students to know. The system also permits reporting on other outcomes such as 
the age-in-grade rate, the attendance rate, the grade promotion rate, and the 
percentage of students meeting graduation requirements. The new standards would 
also allow (and encourage), however, the use of alternative assessment instruments 
and measures that are even more meaningful to other locales and States-especially 
where the content and skills covered in the program's curriculum are not a good match 
to the indicators listed above. MENAES has a "State option" function expressly 
designed to collect data on any number of indicators of student performance that are 
relevant and useful in each State. Examples include: the amount of time spent on 
different subjects, level of English language proficiency, homework assignments 
completed, quality of writing portfolios, number of books read, participation in a 
challenging project, student course taking, student expectations, student attitudes, 
student social behavior, and student achievement as measured by a wide variety of 
new and emerging assessment techniques and instruments. This kind of flexibility is 
consistent with NASDME's belief that an effective performance indicator system 
should utilize information that can be productively used in program improvement 
efforts and in reporting the level of success in meeting State Plan objectives. 

NASDME also recommends that a set of design standards and a strategic 
planning focus be established to guide the long-term development of this system. To 
move from a good idea or beyond good intentions requires thoughtful design as well 
as attention to both costs and benefits. NASDME suggests that the OERI criteria for 
an effective indicator-based performance reporting model (as illustrated in Figure 3) 
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be adopted as the design standard for MENAES. We also suggest that the costs and 
benefits of mandated or extended applications of the system be estimated prior to their 
implementation. 
FIGURE 3: OEM Criteria for a Performance Indicator System 

o Measure the central features of schooling (i.e., inputs, outputs, and implementation); 

o Measure what is actually being taught or considered important for students to know; 

o Provide information that is policy relevant; 

o Focus on the school site; 

o Allow for fair comparison; and 

o Maximize the usefulness of the data collected (i.e., provide information that can be 
readily understood by the audience) and minimize the burden of collecting it. 



The new emphasis on flexibility in the performance indicator system may not 
resolve the tension between accounting for a large number of diverse program 
objectives and accounting for uniform national goals. At present, reducing data from 
multiple sources as well as multiple contexts not only leads to a loss of essential 
information, but also runs the risk of leading to gross misinterpretations. A flexible 
indicator system will, however, improve the accountability of individual State 
programs by having a greater capacity to 1) accurately report results, 2) reward or 
sanction of appropriate entities or practices, and 3) facilitate program improvement in 
each State. Building and sustaining a highly successful program that aspires to 
operate throughout the nation has been and will continue to be accomplished State by 
State. Questions of impact at the national level should be left to well-designed 
national studies. 

Page 22 



The Argument for Accountability that Works 

Recommendation #2: Congress and OME should authorize and encourage the use 
of high quality evaluation designs, especially randomized field experiments, for 
evaluating program effects and program variations . 

The use of higher quality evaluation designs is often mentioned, but is seldom 
supported actively. The obstacles to conducting good "impact" evaluations are many, 
and are often valid. NASDME agrees that these studies are difficult to execute and 
cannot be routinely implemented. But, given the intense interest in discerning 
program effects, NASDME urges that this type of evaluation be encouraged and 
supported in all States where the evaluation design is feasible. 

Support for high quality evaluations has been voiced from researchers as well. 
In An Appraisal of Educational Program Evaluations: Federal State, and Local 
Agencies (1980), principal investigators Robert Boruch and David Cordray made this 
recommendation to the U.S. Congress and the U.S. Department of Education. The 
researchers argued that this type of design is needed if practitioners "want to learn 
how to bring about clear, detectable, changes" and "if the Congress or the Department 
want good estimates of the effects^of programs on children" (Boruch & Cordray, 1980, 
pp. 7-13, 3). NASDME suggests that the use of higher quality evaluation designs, 
where feasible, is in keeping with appropriate professional standards and with public 
interest in evidence of this kind. 

Authorizing high quality program evaluations, however, is only one facet of 
this recommendation. The second, and equally important condition, is encouraging 
States to risk the possible identification of failure or meager results. Finding effective 
ways to help migrant children to achieve school success is not easy, nor does any one 
in America yet have a universal solution. Migrant Education, by its very nature, 
needs systemic success on statewide and interstate bases-a much more difficult and 
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complicated set of tasks. If the States perceive program evaluation as a one-time, high 
stakes trial on total program effectiveness, it is unlikely that they will enter into such 
activities. Rather, the States should be first encouraged to pinpoint and openly 
examine weaknesses in their instructional and support service projects. And then, the 
States must be reassured that they will have an opportunity to respond to evidence of 
weak programming before any final judgements are made about their programs. To 
march toward real solutions, the States must believe that rigorous program evaluation 
as a means toward program improvement is really supported by Congress and the 
federal government. 

To support better evaluation designs to guide program improvement and 
estimate program effects, NASDME asserts that in the absence of fiscal support, the 
effort and risk-taking that is necessary to employ these designs will not occur. As 
mentioned previously, new evaluation efforts not compete for a larger portion of 
an already limited set of resources. Especially, when the competition is "serving 
children." NASDME suggests that additional funds, beyond the regular allocation of 
funds provided by the statute (i.e., Sections 1201 and 1203), be made available to all 
States during the five year period coinciding with re-authorization of the law. The 
new funds would be used to support program evaluations that meet the standards of 
a high quality design. Receipt of these special funds would require approval of a 
State's proposed evaluation design and methodology. In this manner, new, more 
effective, program evaluation designs could be introduced to report on program effects- 
without a decrease in services to children. 

NASDME recognizes that public patience for results neither is nor should be 
infinite. Hopefully, in concert with more effective performance indicator reporting, the 
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results of high quality program evaluations will be able to answer some of the 
fundamental questions about the efficacy of migrant education. NASDME welcomes 
the opportunity to demonstrate the program's effectiveness. NASDME believes its 
members are already operating a number of successful programs, and many more 
"near" successful programs. We would welcome the opportunity and the support to use 
the best methods to guide program improvement and document program 
accomplishments as well as shortcomings. 
Accountability through Monitoring and Compliance in Migrant Education 

Two types of formal monitoring and compliance reviews currently hold individual State 
migrant education programs accountable. They are (1) the financial and compliance audits conducted 
by a State auditor or a public accounting firm and (2) the monitoring and compliance audits conducted 
by staff of the U.S. Department of Education's Office of Migrant Education and/or other branches of 
the federal education enterprise (e.g., the U.S. Inspector General's Office). For the purposes of this 
discussion the term monitoring refers to a program quality review conducted in the field that 
addresses qualitative aspects of the economy and efficiency audit and the program results audit. 

The financial and compliance audit is guided by generally accepted auditing standards 
including, the Government Auditing Standards issued by the U.S. Comptroller General, the Single 
Audit Act of 1984, and the provisions of the Office of Management and Budget Circular A-128-Audit$ 
of State and Local Governments. The financial component is generally conducted by accountants to 
determine whether or not the financial statements of the State are fairly presented in all material 
aspects in relation to the finances of the program. The compliance component is generally conducted 
by accountants (or a qualified internal program auditor) to determine whether the State has complied 
with all laws, regulations, contracts, and grants applicable to the program and as they may affect the 
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financial statements. NASDME believes that this aspect of the monitoring and compliance 
accountability model requires no substantial improvement. 

The second form of monitoring and compliance for accountability is conducted by staff of the 
Office of Migrant Education. In 1991, OME program reviews "focused on aspects of (1) State and local 
administration of the Migrant program; (2) services provided to migratory children; (3) program 
effectiveness; (4) procedures for ensuring interstate and intrastate coordination of services for 
individual children; (5) the operation of the Migrant Student Record Transfer System; and (6) 
identification and recruitment policies and procedures, including the State's system for ensuring the 
accuracy of the data used to determine eligibility" (OME, personal communication, June 18, 1991). 
Eighteen States were reviewed in 1991; 17 are planned for 1992. 

Monitoring and compliance reviews are usually conducted on-site by a team of OME staff. At 
the completion of the review, an informal exit conference is conducted with the State Director. 
Following that meeting, a more formal exit conference is held with the Chief State School Officer or 
designee. A written report is forwarded to the Chief State School Officer within 30 days. The State 
has 60 days to respond as to the action it has taken or will take to carry out the correction action and 
recommendations in the report (OME, personal communication, August 23, 1991). 

NASDME believes the present system has serious flaws. The first weakness deals with setting 
expectations, and the second concerns functionality. Under current procedures, the States have no 
advance knowledge of the specific functions or activities to be monitored or the performance standards 
to be met. Although OME does prepare and use a formal instrument to guide its review team, present 
policy is not to release this information to the States. 3 

NASDME believes that this style of auditing is counter-productive. Rather, States should 
know exactly what is expected of them in order to continually maintain compliance and improve 
programs. Real errors or omissions in a State program can not be disguised easily from well-trained 
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professional monitors. NASMDE suggests that the full dissemination and discussion of an audit 

instrument will do more to stimulate greater accountability than trying to catch States unaware. The 

Association believes that the success of a monitoring and compliance approach should not be measured 

by the number of audit exceptions, but rather by the absence of errors and by the improvements States 

make in order to minimize the number of mistakes or lapses. 

Recommendation #3: OME should disseminate and discuss the program monitoring 
instrument one year in advance of its use in on-site program reviews . 

NASDME recommends th^t OME disseminate the official program monitoring 
instrument to the States at least one year in advance of its use. The monitoring guide 
would then provide the States with a useful tool to help direct (and re-examine) on- 
going efforts in program development and program compliance. By allowing the States 
to preview the guide, to ask questions, and to better prepare for monitoring visits, 
OME will prompt precisely the type of action we believe it hopes to encourage within 
every State program. 

The dissemination of the guide should be accompanied by written and oral 
presentations on 1) the findings fro*n the previous year's monitoring activities and 2) 
on those aspects of the program that OME will consider a priority area(s) for review 
in the upcoming year. These briefings can be informative, as well as instructive. 
NASMDE recommends that OME briefings share (or facilitate the sharing of) the best 
practices and/or solutions to common problems that they identified throughout their 
monitoring of State programs. NASMDE suggests that OME regularly schedule a 
session during its Annual Meeting with State Directors to conduct these briefings and 
to disseminate relevant materials. 
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In terms of functionality, a major limitation in the current program monitoring scheme is that 

it relies solely on an infrequent, single-event, high-stake experience. Why this characterization? 

Activities become routine at the state level when they occur on a regular annual basis. Program 

monitoring visits are perceived as infrequent because they typically occur at three year intervals. 

Because the time interval between monitoring visits is so long for most States, the Directors and staff 

members often perceive the process as a single event that begins and ends in the space of one week. 

It i* not perceived as a part of a larger, on-going cycle of corrective feedback. And, for most migrant 

educators, program reviews-as currently constructed-represent an important, stressful, high-stakes 

experience-with both the positive a.id negative results reported to the Chief State School Officer. 

Unfortunately, the temptation for State Directors becomes to avoid exposing program weaknesses and 

survive unscathed, even if this attitude prohibits the very learning and growth that is desired. The 

risks of doing otherwise are too great. 

Recommendation #4: Congress and OME should support an optional peer 
monitoring and compliance review program . 

To improve the functionality of program monitoring and compliance reviews, 
Congress and OME should consider supporting an optional "peer review" for the 
migrant education program. NASDME offers the following guidelines. The peer 
review would focus on program performance. The peer monitoring and compliance 
review option would enable a State Director to identify three peers to conduct a week- 
long review of the State's program using the OME monitoring instrument as a guide. 
The peer review team would be asked to follow protocols similar to that used by OME 
monitoring teams. At the end of the review, the peer group would conduct a formal 
exit interview with the State Director and would later follow up their presentation of 
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findings in a confidential written report. Dissemination of the report would be 
contingent upon the State Director's consent. 

As a form of interstate collaboration, peer reviews could be funded under 
Section 1203 and facilitated by the existing Program Coordination Centers. 
Participation in the peer monitoring and compliance review process would be optional. 
A peer review would be conducted only upon the request of the State Director. The 
reviews could be scheduled for up to one and one-half years before the next scheduled 
OME review or one and one-half years after the most recent monitoring visit. Those 
State Directors interested in serving as peer reviewers would receive an orientation 
from OME and follow-up training on effective peer review processes. 

The benefits of this approach are three-fold. First, the frequency of program 
monitoring can be increased and thus strengthen the cycle of corrective feedback (and 
the perception of its value). In this format, program monitoring would afford two 
visits by external monitors in each three year cycle. The first visit in the cycle would 
be for the direct benefit of the State Director to assess program development and 
compliance. The peer review team would examine the program, make suggestions, and 
highlight any needed corrective action(s). Thus, the monitoring cycle would now 
provide for a low-risk "learning" experience that would better help to identify 
operational difficulties and to evaluate potential solutions with experienced colleagues. 
The State Director would then work to make these improvements with the knowledge 
that the official OME visit would follow as scheduled. 

The second major benefit of this approach is that State Directors have 
immediate access to respected peers who understand the problems first-hand and who 
are in a position to criticize constructively without penalties induced. The third 
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benefit of the peer review process is that it would be an invaluable learning experience 
for the members of each review team. It will come as no surprise to anyone if the 
individuals, who have been monitors and monitored, report that they learned more and 
made greater changes in their own programs through the experience of reviewing 
another State with their peers. Over time, the cumulative effect of the process will 
sharpen NASDME's collective wisdom. 

This option can be realized only with OME's active support. OME must be 
willing to play a very difficult role, a role that empowers State Directors to work 
independently yet toward the same goals articulated by OME. To be successful, OME 
will have to facilitate peer reviews and simultaneously share the ownership of the 
process with the State Directors. The ongoing official program reviews would continue 
to provide OME an opportunity to publicly monitor and accredit the agency's efforts 
in the provision of services, to review management control systems that influence 
program quality, and to examine compliance with program standards or regulations. 
Accountability through Professionalism in Migrant Education 

In discussing the third and final accountability model that operates in migrant education, 
NASDME recognizes its own responsibility. The desire to innovate, to develop, and to implement high 
quality services for migrant children is rooted in the inspiration and commitment that migrant 
educators bring to the profession and that we demand of each other. The desire to persist, until the 
equality of educational opportunity is secured for all migrant children, is sustained largely by a set 
of beliefs about the promise of migrant children and the importance of helping them to realize their 
full potential. The desire to embrace these challenges and to take risks to serve migrant children 
better is part of a culture that must exist within the community of migrant educators--if the program 
is to succeed. Accountability based on technical solutions (i.e., reporting numbers and compliance) can 
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only function if accompanied by accountability driven by professionalism. A good accountability 
system for migrant education, therefore, would also include the means for progressive change in the 
professional roles, norms, and values of migrant educators. In fact, in many fields of endeavor, 
observers of effective non-profit organizations consider this approach to be the best accountability 
model (Drucker, 1990). 

One vehicle that helps migrant educators impose accountability from within its ranks is the 
National Association of State Directors of Migrant Education itself NASDME affords the individual 
States an opportunity to come together and, amongst ourselves, set standards and expectations that 
we will agree to and by which we will operate our programs. NASDME believes, however, that the 
full potential of the association's collective energy, resources, and commitment has yet to be achieved. 
We believe that greater effort to advance the collective capacity of NASDME-to improve its members 
and to improve services by self-determination-is needed. Thus, if this kind of fundamental 
accountability is recognized as valuable, it follows, that efforts should be made to promote this capacity 
within NASDME. 

Recommendation #5: NASDME should actively help its members improve each other 
and take responsibility for assessing the quality of the migrant education program at 
an annual leadership retreat . 

NASDME activities are conducted through a variety of means. Individual 
State Directors are encouraged to participate in annual meetings, at conferences, in 
workshops, on committees, on special interstate projects, and, most importantly, in the 
implementation of NASDME-supported activities. Only rarely, however, can the total 
membership step back from the day-to-day details of program operations to reflect on 
quality and long term improvements within and across the States. 

Each year NASDME meets twice. In each meeting, the agenda is crammed 
with reports and presentations, and little time is left for real communication between 
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the members. Time for open dialogue and reflection, if it exists, occurs in small group 
settings in after-hour discussions. Given the few opportunities NASDME has to meet 
as a whole, and the amount of regular business that needs to be conducted, this 
situation is not likely to change if the time allotted for full Association meetings 
remains the same. 

NASDME recommends that Congress and OME support an annual leadership 
retreat in which State Directors evaluate individual and organizational effectiveness. 
The retreat would serve to stimulate reflection on the assumptions, norms, and values 
that support professional roles and practice in migrant education. The retreat would 
be specifically designed to create a climate of trust and support, where face-to-face 
communication on issues of personal leadership and on areas of program strength and 
weakness could be examined in full. The soon-to-be-published Descriptive Study of the 
Chapter 1 Migrant Education Program would provide a perfect focus for the first such 
event. Serious reflection on the results of the study could provide NASDME with an 
opportunity to consider needed changes in goals and expectations, changes in 
individual and organizational competencies, and changes in action strategies to fulfill 
the aims of migrant education. Accountability in this context is future-oriented and 
encourages efforts to change and grow, rather than solely rewarding or sanctioning 
past performance. NASDME believes that it is through this type of learning that it 
could stimulate a self-renewing process of reflection, problem-solving, and action that 
leads to accountability of the most fundamental kind. 

THE PATH TOWARD IMPROVED ACCOUNTABILITY 

In this position paper, we have described and examined the emergent definition and forms of 

educational accountability. We did so for five central reasons. First, because an accountability system 
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can work only when the key parties have agreed upon who is being held accountable, and for what 
NASDME strongly asserts that any form of educational accountability must be directed toward the 
relationship between the individual State and the U.S. Department of Education-as expressly 
negotiated in an approved Annual Program Plan. 

Second, various forms of accountability mechanisms differ, appropriately so, for different 
contexts. The accountability system that is developed for migrant education should reflect a sensible 
combination of accountability models, models that can work in the context of a State-administered 
education program. NASDME believes that three of these models-performance reporting, monitoring 
& compliance, and professionalism-do hold great promise, and we recommend that we improve upon 
these complimentary approaches. The value and potential of these models, operating synchronously, 
will not be realized if we do not work together to develop them. 

Third, an effective system must be flexible and include the capacity to foster program 
improvement. NASDME believes that the old "single indicator" performance reporting mechanisms 
(i.e., TIERS) are inadequate for this task. NASMDE recommends, therefore, that the migrant 
education program, out of necessity and choice, be empowered to explore a new generation of 
alternative performance indicators that will better help to guide program planning and improvement 
(in addition to serving the other functional aspects of accountability). 

Fourth, the demand for proof of results at the national level may actually diminish the 
possibility of instituting rigorous program evaluations. If there are questions about the effects of a 
State program, then the authorities need to be willing to support the kind of high quality evaluation 
designs that can answer that question. Individual States can produce data on the effects of their 
programs-if bearing the costs of such inquiry are encouraged and supported by the lawmakers and 
the federal education bureaucracy. If we are serious about providing services that work then we must 
be willing to implement evaluation practices that do the same. If we want to conduct research or 
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inquiry at the national level, we repeat, the audience that awaits this information will be better served 
by a carefully designed and implemented national study. 

Finally, we recognize the value of incorporating the human factor, in a meaningful way, into 
the accountability system. Migrant educators serve migrant children. Migrant educators will 
determine whether to provide services that simply satisfice or whether to try to intervene with 
maximum effect. Unless migrant educators hold themselves, their colleagues, staff, and organizations 
accountable, a truly effective accountability system will not be realized. Migrant educators must be 
empowered to routinely ask and then act on the question; does this use of resources really serve to 
help a migrant child succeed in school? Without the conscience of professionalism, rules, regulations, 
and reporting are all empty strategies. NASDME offered peer monitoring and compliance reviews and 
professional development opportunities for State Directors as two simple ways to begin to expand and 
to improve upon this element of an accountability system that works. 
A FINAL THOUGHT 

In The Fifth Discipline: The Art & Practice of The Learning Organization, Peter Senge (1990) 
identifies five disciplines vital to building a learning organization: personal mastery, mental models, 
building shared vision, team learning, and-the fifth discipline-systems thinking. While each of the 
dimensions is important, Senge places "systems thinking" as the cornerstone discipline, a first among 
equals. He argues that systems thinking is a useful tool to examine the dynamic complexity of 
organizational life, especially when cause and effect are not close in time and space and obvious 
interventions do not produce expected outcomes (1990). He also posits that systems thinking is a 
proactive way to discern high from low leverage opportunities for change toward desired outcomes. 

It is this principle of leverage that we wish the reader to consider in the context of the migrant 
education program. Leverage is seeing where actions and changes in structures lead to significant, 
enduring improvements (Senge, 1990). Senge notes that leverage "often follows the principle of 
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economy of means; where the best results come not from large-scale efforts but from small well-focused 

actions" (p. 114). He further explains the idea of leverage by way of an example. 

Buckminster Fuller had a wonderful illustration of leverage that also served as his 
metaphor for the principle of leverage-the "trim tab." A trim tab is a small "rudder 
on the rudder" of a ship. It is only a fraction of the size of the rudder. Its function is 
to make it easier to turn the rudder, which, then, makes it easier to turn the ship. 
The larger the ship, the more important is the trim tab because a large volume of 
water flowing around the rudder can make it difficult to turn (p. 64). 

Throughout the past twenty-five years, the federal role in education has been to advocate for 
the interests of those children who are most in need: the poor, the nandicapped, racial and ethnic 
minorities, and those children who by their life's circumstance were often excluded from the full 
benefits of the public education system. From NASDME's perspective, the 1990s have witnessed the 
emergence of a new federal role in education. In addition to its traditional functions, the federal role 
in education now incorporates a growing and active interest in promoting education reform. 

In thinking about this new context, it is useful to employ Buckminster Fuller's metaphor. The 
federal government's role in public education has been to ensure that there was enough room on the 
ship for all children to have a seat (i.e., equal educational opportunity). The new federal role envisions 
using federal funds and resources to help steer the ship (Doyle, 1991). Accountability systems in this 
context should be designed to act as a trim tab, to help both the federally-funded programs and the 
local public schools move forward together. 

NASDME views the Chapter 1 Migrant Education Program as an important contribution to 
both of the federal government's goals in education: equality of educational opportunity and education 
reform. We believe that a migrant education program operating with an effective accountability 
system can help to increase progress toward these two goals. In this position paper, we have 
presented five important improvements to the existing accountability system. The changes do not call 
for wholesale transformations, but rather propose small well-focused shifts in emphasis in current 
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accountability policies and practices. We believe that the proposed changes will help to create a high 
leverage accountability system, and thus conceived and implemented, one that will workl 
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NOTES 



1. The National Association of State Directors of Migrant Education (NASDME) was officially created in 1975. NASDME is 
a voluntary Association with no memborship dues. All State Directors of Migrant Education are eligible for active membership 
and entitled to vote in meetings of the full membership. The Association meets at least twice each year. NASMDE's primary 
purpose is to provide a vehicle for interstate communication and cooperation in the quest for quality and continuity in programs 
for migrant children. NASDME also provides a means of strengthening the leadership role of State Directors in both intrastate 
and interstate activities. The Association annually elects three officers; President-Elect, President, and Secretary -Treasurer. 

2. The percentage of students with norm-referenced test scores is an estimate based on the counts reported in the FY'91 MSRTS 
Participation Report. The frequency count was limited to five major norm-referenced tests. The number of students with tost 
score on different instruments was not counted. In FT91, 107,412 migratory children were reported with a point-in-time test 
score in Reading. In terms of matched test scores, the MSRTS reported that 13,074 formerly migratory children had both a pre- 
test and post-test score. Using the MSRTS Student Distribution Report the estimated total number of students in grades two 
through twelve is 550,000 for all children, and 450,000 excluding "residency only" children. 

3. The Office of Migrant Education, U.S. Department of Education, disseminated the State Education Agency Monitoring Guide 
at the FY92 Annual Directors Meeting that was sponsored by OME. It is unclear as to whether this practice will become official 
policy. 
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